Overall Analysis

Objective: General Analysis

1. Set Up

README

In general, the data sets are relatively large, and take up a large amount of memory after loading. It is recommended that after loading, that the data sets be subsetted or summarised, and unused data frames to be removed.

The code is shown in the next three tabs to show any changes that may have been made. In general, no columns or rows are removed. Filtering occured in the next step.

General

Loading 1850

Loading 1880

Loading 1910

Combining all time periods, applying occ_modifier

## Parsed with column specification:
## cols(
##   year = col_double(),
##   age = col_double(),
##   sex = col_character(),
##   race = col_character(),
##   city = col_character(),
##   enumdist = col_double(),
##   ward = col_double(),
##   occstr = col_character(),
##   occstr_mod = col_character(),
##   occ = col_double(),
##   occ_label = col_character(),
##   occ_cat_label = col_character(),
##   occ1950 = col_double(),
##   occ1950_label = col_character(),
##   occ1950_cat_label = col_character(),
##   ind1950 = col_double(),
##   ind1950_label = col_character(),
##   ind1950_cat_label = col_character()
## )

shp Files

## Reading layer `Ward_1850_BK' from data source `C:\Users\Clinton\Documents\GitHub\hnyc_occupations\Data\shpfiles\bk_shapefiles\Ward_1850_BK.shp' using driver `ESRI Shapefile'
## Simple feature collection with 11 features and 3 fields
## geometry type:  POLYGON
## dimension:      XY
## bbox:           xmin: -8240555 ymin: 4958954 xmax: -8226921 ymax: 4969322
## epsg (SRID):    3857
## proj4string:    +proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +wktext +no_defs

## Reading layer `Ward_1850_MN' from data source `C:\Users\Clinton\Documents\GitHub\hnyc_occupations\Data\shpfiles\mn_shapefiles\Ward_1850_MN.shp' using driver `ESRI Shapefile'
## Simple feature collection with 19 features and 3 fields
## geometry type:  POLYGON
## dimension:      XY
## bbox:           xmin: -8239443 ymin: 4968339 xmax: -8227670 ymax: 4994306
## epsg (SRID):    3857
## proj4string:    +proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +wktext +no_defs

## Reading layer `ED_1880_S4_BK' from data source `C:\Users\Clinton\Documents\GitHub\hnyc_occupations\Data\shpfiles\bk_shapefiles\ED_1880_S4_BK.shp' using driver `ESRI Shapefile'
## Simple feature collection with 250 features and 152 fields
## geometry type:  POLYGON
## dimension:      XY
## bbox:           xmin: -8240479 ymin: 4958979 xmax: -8226015 ymax: 4973979
## epsg (SRID):    3857
## proj4string:    +proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +wktext +no_defs

## Reading layer `ED_1880_MN' from data source `C:\Users\Clinton\Documents\GitHub\hnyc_occupations\Data\shpfiles\mn_shapefiles\ED_1880_MN.shp' using driver `ESRI Shapefile'
## Simple feature collection with 662 features and 41 fields
## geometry type:  POLYGON
## dimension:      XY
## bbox:           xmin: -8239612 ymin: 4968158 xmax: -8227670 ymax: 4994306
## epsg (SRID):    3857
## proj4string:    +proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +wktext +no_defs

## Reading layer `Brooklyn_1910' from data source `C:\Users\Clinton\Documents\GitHub\hnyc_occupations\Data\shpfiles\bk_shapefiles\Brooklyn_1910.shp' using driver `ESRI Shapefile'
## Simple feature collection with 1112 features and 1 field
## geometry type:  MULTIPOLYGON
## dimension:      XY
## bbox:           xmin: 1825539 ymin: 556929.4 xmax: 1841193 ymax: 576192.4
## epsg (SRID):    NA
## proj4string:    +proj=aea +lat_1=29.5 +lat_2=45.5 +lat_0=37.5 +lon_0=-96 +x_0=0 +y_0=0 +datum=NAD83 +units=m +no_defs

## Reading layer `Manhattan_1910' from data source `C:\Users\Clinton\Documents\GitHub\hnyc_occupations\Data\shpfiles\mn_shapefiles\Manhattan_1910.shp' using driver `ESRI Shapefile'
## Simple feature collection with 1480 features and 1 field
## geometry type:  MULTIPOLYGON
## dimension:      XY
## bbox:           xmin: 1823260 ymin: 568821.3 xmax: 1831540 ymax: 591889.7
## epsg (SRID):    NA
## proj4string:    +proj=aea +lat_1=29.5 +lat_2=45.5 +lat_0=37.5 +lon_0=-96 +x_0=0 +y_0=0 +datum=NAD83 +units=m +no_defs

2. Analysis of Demographics

Age

General Trends

##    vars       n  mean    sd median trimmed   mad min max range skew
## X1    1 2712978 33.04 12.75     30   31.74 13.34  16 126   110 0.85
##    kurtosis   se
## X1     0.24 0.01

Yearly Trends

## 
##  Descriptive statistics by group 
## group: 1850
##       vars      n  mean    sd median trimmed   mad min max range skew
## age      1 185032 32.04 11.11     30    30.9 10.38  16 126   110 1.01
## year*    2 185032  1.00  0.00      1     1.0  0.00   1   1     0  NaN
##       kurtosis   se
## age       1.06 0.03
## year*      NaN 0.00
## -------------------------------------------------------- 
## group: 1880
##       vars      n mean    sd median trimmed   mad min max range skew
## age      1 705712 33.4 12.97     30   32.14 13.34  16 108    92 0.78
## year*    2 705728  2.0  0.00      2    2.00  0.00   2   2     0  NaN
##       kurtosis   se
## age       0.03 0.02
## year*      NaN 0.00
## -------------------------------------------------------- 
## group: 1910
##       vars       n mean    sd median trimmed   mad min max range skew
## age      1 1822234   33 12.82     30   31.68 13.34  16 104    88 0.86
## year*    2 1822234    3  0.00      3    3.00  0.00   3   3     0  NaN
##       kurtosis   se
## age       0.26 0.01
## year*      NaN 0.00

Sex

General Trends

Percent of Labor Force

Race

General Trends

Percent of Labor Force

City

General Trends

Percent of Labor Force

3. Top 10s Analysis

Occupation

Datatable of General Counts

Overall

• Across years, Operative and kindred workers (n.e.c.), Managers, officials, and proprietors (n.e.c.), Private household workers (n.e.c.) and Laborers (n.e.c.) appear to be the occupations which consistently are the most common

1850

1880

1910

Graph of General Counts

General Trends

Percent of Labor Force

Occupation Categories

Datatable of General Counts

Overall

• Across years, Operative and kindred workers (n.e.c.), Managers, officials, and proprietors (n.e.c.), Private household workers (n.e.c.) and Laborers (n.e.c.) appear to be the occupations which consistently are the most common

1850

1880

1910

Graph of General Counts

General Trends

Percent of Labor Force

Industry

Datatable of General Counts

Overall

1850

1880

1910

Graph of General Counts

General Trends

Percent of Labor Force

Industry Categories

Datatable of General Counts

Overall

1850

1880

1910

Graph of General Counts

General Trends

Percent of Labor Force

4. Maps of Counts

1850

Leaflet Map of Counts

Distributions of Occupation

Occupation

Normalised by size of each ward

Grouped

Normalised and grouped by size of each ward

Distributions of Race

Race

Normalised by size of each ward

Grouped

Normalised and grouped by size of each ward

1880

Leaflet Map of Counts

Distributions of Occupation

Occupation

Normalised by size of each enumeration district

Grouped

Grouped and Normalised by size of each enumeration district

Distributions of Race

Race

Normalised by size of each enumeration district

Grouped

Grouped and Normalised by size of each enumeration district

1910

Leaflet Map of Counts

Distributions of Occupation

Occupation

Normalised by size of each enumeration district

Grouped

Grouped and Normalised by size of each enumeration district

Distributions of Race

Race

Normalised by size of each enumeration district

Grouped

Grouped and Normalised by size of each enumeration district

5. Analysis of Changing Trends

OCC

Graph of Top 10 Occupations

Graph of Top 10 Occupations, Normalised

Maps of Top Occupation

Operative and kindred workers (n.e.c.)

Operative and kindred workers (n.e.c.)

Operative and kindred workers (n.e.c.) Normalised

Managers, officials, and proprietors (n.e.c.) (n.e.c.)

Managers, officials, and proprietors (n.e.c.)

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

Managers, officials, and proprietors (n.e.c.) Normalised

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

## Warning in grid.Call(C_textBounds, as.graphicsAnnot(x$label), x$x, x$y, :
## font width unknown for character 0x41

Occ Categories

Graph of Top 10 Occupation Categories

Graph of Top 10 Occupation Categories, Normalised

Industry

Graph of Top 10 Industries

Graph of Top 10 Industries, Normalised

Industry Categories

Graph of Top 10 Industries Categories

Graph of Top 10 Occupation Categories, Normalised

6. Heatmaps of Combinations

Occupation

Industries